Exploring Task Structure for Brain Tumor Segmentation From Multi-Modality MR Images

Problem Statement

Brain tumor segmentation, which aims at segmenting the whole tumor area, enhancing tumor core area, and tumor core area from each input multi-modality bioimaging data, has received considerable attention from both academia and industry.

However, the existing approaches usually treat this problem as a common semantic segmentation task without taking into account the underlying rules in clinical practice.

Clinical Insights

In reality, physicians tend to discover different tumor areas by weighing different modality volume data. Also, they initially segment the most distinct tumor area, and then gradually search around to find the other two. We refer to:

1. Task-Modality Structure: The first property where different modality data have different importance for different tasks.

2. Task-Task Structure: The second property where physicians initially segment the most distinct tumor area, then gradually search for others.

Proposed Method: TSBTS Net

Based on these insights, we propose a novel task-structured brain tumor segmentation network (TSBTS net).

To explore the task-modality structure: We design a modality-aware feature embedding mechanism to infer the important weights of the modality data during network learning.

To explore the task-task structure: We formulate the prediction of the different tumor areas as conditional dependency sub-tasks and encode such dependency in the network stream.

Experimental Results

Experiments on BraTS benchmarks show that the proposed method achieves superior performance in segmenting the desired brain tumor areas while requiring relatively lower computational costs, compared to other state-of-the-art methods and baseline models.

By exploring both task-modality and task-task structures inspired by clinical practice, TSBTS net provides an effective and efficient solution for multi-modality brain tumor segmentation.